Workshop  Session  #2: 

Human  Effectiveness  and  Embedded  Training  Environments 

Summary  of  Discussion 


Dr.  Dee  Andrews  (USA)  and  Dr.  Jan  Roessingh  (NED)  facilitated  this  workshop  session  centered  on 
embedded  training  environments  and  specifically  on  training  management  and  distributed  embedded 
training  modes.  Three  presentations  were  provided  for  discussion: 

•  Training  effectiveness  of  embedded  training  in  a  (multi-)  fighter  environment  -Jan  Roessingh, 
Ph.D.  (NED)  National  Aerospace  Laboratory;  and  LTC  Gerbe  Verhaaf  (NED) 

•  Developing  Effective  Embedded  Training  -  Jan  Cannon-Bowers,  Ph.D.  (USA),  University  of 
Central  Florida  Institute  for  Simulation  and  Training 

•  Leveraging  Embedded  Training  Systems  to  Build  Higher  Level  Cognitive  Skills  in  Warfighters  - 
Mica  Endsley,  Ph.D.  (USA),  SA  Technologies;  Jennifer  Riley,  Ph.D.  (USA),  SA  Technologies; 
and  Laura  Strater,  Ph.D.  (USA),  SA  Technologies 

Workshop  Exercise  #2:  Mindmapping  Embedded  Virtual  Simulation  training  management 

For  this  session,  the  mindmapping  exercise  focused  on  EVS  training  management  and  the  primary  and  the 
five  secondary  questions  below  were  intended  to  focus  and  energize  the  discussion: 

•  How  do  you  manage  training  in  EVS? 

o  What  are  viable  instructional  strategies  for  EVS? 
o  What  categories  of  metrics  can  be  used  to  manage  EVS? 
o  How  do  you  manage  training  sessions? 

o  How  do  you  envision  tasks  currently  accomplished  by  “white  cell”  personnel? 
o  How  will  you  manage  training  in  different  modes  (homestation,  in  transit,  in  theater)? 

The  mindmaps  shown  below  are  the  group  products  from  Exercise  #2  on  EVS  training  management: 
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Figure  1:  Group  1  Mindmap  for  Exercise  #2 
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Figure  2:  Group  2  Mindmap  for  Exercise  #2 
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Figure  3:  Group  3  Mindmcip  for  Exercise  #2 

For  this  session,  the  mindmapping  exercise  focused  on  EVS  training  management  and  the  primary  and  the 
five  secondary  questions  below  were  intended  to  focus  and  energize  the  discussion.  Below  is  a  summary 
of  the  outcomes  of  the  joint  results  of  the  three  discussion  groups: 

What  are  viable  instructional  strategies  for  EVS? 

The  simplest  instructional  strategy  seems  to  just  confronting  the  trainee  with  a  scenario  (Scenario  Based 
Training)  using  EVS,  i.e.  involving  some  degree  of  simulation.  Thus,  constructing  an  appropriate  scenario 
is  the  minimum  effort  that  needs  to  be  made  in  terms  of  devising  an  instructional  strategy.  This  however, 
can  be  a  major  effort  in  the  complex  skill  domains  we  are  looking  at  for  ET  in  the  military. 

Additional  instructional  strategies  that  were  mentioned  are:  whole-task  training  with  injection  of  virtual 
entities  into  the  training  scenario,  instructional  strategies  based  on  intelligent  tutoring  (replacing  the 
human  instructor),  instruction  strategies  that  are  based  on  remotely  monitoring  of  the  exercise  and 
providing  instructor  feedback  via  radio  (e.g.  emphasis  training),  adaptive  training,  in  which  parameters  of 
the  training  environment  (level  of  complexity,  speed,  etc.)  will  change  depending  on  progress  of  the 
trainee. 

It  can  helpful  if  ET  developers  structure  Advanced  Organizers  for  the  training.  David  Ausubel  (????) 
posited  that  it  is  important  to  present  to  learners  a  cognitive  scaffold  of  the  material  to  be  learned  before 
they  begin  learning  the  details.  Ausubel’s  advance  organizer  can  best  be  classified  as  a  deductive  method. 
Deductive  methods  or  reasoning  provide  the  rule  to  follow  then  the  example  leading  to  the  correct  answer 
or  learning  (Mayer,  2003).  This  is  opposite  from  inductive  methods  or  reasoning  that  provides  the  example 
to  follow  then  the  rule. 
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Advance  organizers  are  also  highly  useful  in  the  process  of  transferring  knowledge.  Because  of  the 
deductive  reasoning,  students  are  able  to  use  the  rule  then  the  example  for  learning  to  occur.  Mayer  writes 
in  his  text,  "...the  effects  of  advance  organizers  should  be  most  visible  for  tests  that  involve  creative 
problem  solving  or  transfer  to  new  situations,  because  the  advance  organizer  allows  the  learner  to  organize 
the  material  into  a  familiar  structure"  (Mayer,  2003). 

-  Mayer,  R.  (2003)  Learning  and  Instruction.  New  Jersey:  Pearson  Education,  Inc 

Part  of  the  instruction  strategy  could  be  that  while  being  deployed,  e.g.  in  Afghanistan,  that  trainees  can 
“reach  back”  to  their  home  country,  e.g.  a  school  or  any  other  training  resource. 

Finally,  cross-training  was  mentioned,  i.e.  team  training  in  which  team  members  take  on  other  team 
members’  roles  and  responsibilities,  resulting  in  shared  awareness  of  the  teams  tasks,  roles, 
responsibilities  and  capabilities. 

An  important  instructional  strategy  that  was  not  mentioned  by  any  of  the  participants  was  part-task 
training.  Can  we  conclude  that  part-task  training  is  not  a  viable  instructional  strategy  for  ET,  because  ET, 
implemented  on  a  weapon  platform,  is  naturally  bound  to  a  whole-task  context? 

If  so,  the  viability  of  instructional  strategies  seems  to  depend  on  their  suitability  for  imposing  the  strategy 
on  the  whole-task  context. 

More  general  remark  were: 

(1)  that  the  selection  of  training  media  should  not  be  done  before  an  appropriate  instructional  strategy 
has  been  selected. 

(2)  that  the  appropriate  instructional  strategy  depends  on  the  instructional  objectives  and  the  training 
program  as  a  whole. 

What  categories  of  metrics  can  be  used  to  manage  EVS? 

Distinction  was  made  between  (1)  outcome  measures  {how  well  the  mission  /  task  goals  have  been 
achieved),  and  (2)  process  measures  {how  mission  /  task  goals  were  or  were  not  achieved,  e.g.  frequencies 
of  certain  behavior,  communication,  mutual  support  in  teamwork,  etc.). 

At  the  same  time,  because  ET  relates  to  complex  task  performance,  there  is  a  need  for  multiple  measures, 
covering  multiple  aspects  of  behavior  (behavioral  measures,  psycho-physiological  measures).  In  addition, 
the  need  for  analysis  of  performance-  or  learning  curves  was  mentioned,  an  issue  that  also  has  been 
considered  under  the  heading  of  training  management  /  learning  management  systems. 

Behavioral  measures  that  were  mentioned  were  eye-movement  measures  (EPOG,  blink  frequencies,  pupil 
diameter).  It  was  realized  that  certain  performance  measures  may  not  be  applicable  to  ET,  for  example 
those  collected  by  a  human  instructor  for  feeding  back  immediately  to  the  trainee,  because  human 
instructors  are  not  always  present  in  an  ET  concept,  which  highlights  the  need  for  Intelligent  Tutoring 
Systems.  Finally,  the  need  was  expressed  for  getting  a  measure  (cues)  for  the  level  of  expertise  of  a  trainee 
before  starting  the  ET  session,  in  order  to  set  the  appropriate  scenario. 

How  do  you  manage  training  sessions? 

ET  seems  to  diminish  the  role  of  the  human  instructor,  simply  because  simulation  hard/software  is 
attached  to  real  systems  hardware/software,  resulting  in  a  training  medium  that  most  often  doesn’t  provide 
physical  space  for  a  human  instructor.  Thus  feedback  from  a  human  instructor  will  often  be  delayed  to 
some  point  after  the  mission  exercise,  i.e.  during  an  After  Action  Review  (AAR)  or  mission  debriefing. 
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The  dual  role  of  the  supervisor  was  mentioned,  probably  meaning  that  he  is  both  supervisor  of  the  trainee 
(senior  person)  and  has  an  instructional  role  in  deciding  what  to  do  next. 

An  obvious  remark  is  that  the  instructor  could  be  replaced  by  an  intelligent  tutoring  system.  At  a  meta¬ 
instruction  level,  the  training  sessions  could  be  managed  by  a  Learning  Management  System  (LMS). 


How  would  it  interface  with  the  EVS? 

Functional  requirements  imposed  upon  such  system  are: 

-  record  keeping  of  exposure  to  specific  training  events  (counting); 

-  scheduling  trainee  events  in  order  to  optimize  the  learning  process; 

-  scenario  configuration  management  (which  scenarios  are  updated  and  ready  for  use); 

-  persistent  trainee  profiling  (possibly  throughout  his  career)  on  the  basis  of  performance  measures. 


How  do  you  envision  tasks  currently  accomplished  by  “white  cell”  personnel? 

White  cell  personnel  is  the  support  personnel  during  live  exercises,  e.g.  fulfilling  role  playing  tasks, 
exercise  performance  scoring,  refereeing,  control  of  opposing  forces  (SAFs),  etc.  Some  participants 
hypothesized  that  ET  implementations  could  make  white  cell  Personnel  superfluous  by  automating  their 
tasks.  This  is  certainly  dependent  on  the  level  of  intelligence  and  autonomy  of  the  various  entities  in  the 
ET  virtual  scenario. 

The  question  was  raised  whether  exercise  planning  software  packages  exist,  presumably  to  allocate  tasks 
in  ET  scenarios  to  white  cell  personnel,  or  rather:  Do  scenario  management  systems  for  real  or  simulated 
exercises  currently  support  entities  for  white  cells? 

How  will  you  manage  training  in  different  modes  (homestation,  in  transit,  in  theater)? 

This  question  invoked  no  reactions  by  the  35  participants,  certainly  because  the  session  was  too  short  to 
address  all  five  questions,  and  this  was  the  last  question. 
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